Conditional Random Fields, Planted Constraint Satisfaction and Entropy Concentration

This paper studies a class of probabilistic models on graphs, where edge variables depend on incident node variables through a fixed probability kernel. The class includes planted constraint satisfaction problems (CSPs), as well as more general structures motivated by coding and community clustering problems. It is shown that under mild assumptions on the kernel and for sparse random graphs, the conditional entropy of the node variables given the edge variables concentrates around a deterministic threshold. This implies in particular the concentration of the number of solutions in a broad class of planted CSPs, the existence of a threshold function for the disassortative stochastic block model, and the proof of a conjecture on parity check codes. It also establishes new connections among coding, clustering and satisfiability. ∗ Department of Electrical Engineering and Program in Applied and Computational Mathematics, Princeton University, Email: [email protected] † Departments of Electrical Engineering and Statistics, Stanford University, Email: [email protected]